Markerless Augmented Reality with Light Source Estimation for Direct Illumination

We present a markerless augmented reality system with intuitive positioning and realistic direct lighting of virtual objects. Our approach exploits a two camera system consisting of a TV-camera and a fish-eye camera. The TVcamera captures the video for the augmentation while the fish-eye camera observes the upper hemisphere to track the light sources. Accordingly, direct lighting can be used to render virtual objects. Additionally, we propose an easy method to place virtual objects in the scene and to compute the shadows of these objects. The proposed tracking methods overcome the limitations of many systems that need markers in the scene for augmentation or need additional equipment in the scene to reconstruct illumination. Our approach will keep the augmented scene unaffected.